Cautionary tales in the interpretation of clinical studies involving older persons

The care of patients 65 years or older presents a challenge for evidence-based medicine. Such patients are underrepresented in clinical trials, are more vulnerable to treatment-induced harm, and often are unable to fully participate in treatment decisions. We outline several cautionary themes in the interpretation of clinical studies of therapeutic interventions involving older persons as they apply to processes of everyday clinical decision making. In particular, we focus on issues of study design and quality of evidence, choice of outcome measures, missing outcome data, assessment of potential harm, quantifying treatment effects in individual patients (and adjusting these for effect modifiers and reduced life expectancy), eliciting patient values and preferences, prioritizing therapeutic goals and selection of treatments, and assisting patients in adhering to agreed therapeutic regimens. ©2010 American Medical Association. All rights reserved

From the Trenches: A Cross-Sectional Study Applying the GRADE Tool in Systematic Reviews of Healthcare Interventions

Background: GRADE was developed to address shortcomings of tools to rate the quality of a body of evidence. While much has been published about GRADE, there are few empirical and systematic evaluations. Objective: To assess GRADE for systematic reviews (SRs) in terms of inter-rater agreement and identify areas of uncertainty. Design: Cross-sectional, descriptive study. Methods: We applied GRADE to three SRs (n = 48, 66, and 75 studies, respectively) with 29 comparisons and 12 outcomes overall. Two reviewers graded evidence independently for outcomes deemed clinically important a priori. Inter-rater reliability was assessed using kappas for four main domains (risk of bias, consistency, directness, and precision) and overall quality of evidence. Results: For the first review, reliability was: k = 0.41 for risk of bias; 0.84 consistency; 0.18 precision; and 0.44 overall quality. Kappa could not be calculated for directness as one rater assessed all items as direct; assessors agreed in 41 % of cases. For the second review reliability was: 0.37 consistency and 0.19 precision. Kappa could not be assessed for other items; assessors agreed in 33 % of cases for risk of bias; 100 % directness; and 58 % overall quality. For the third review, reliability was: 0.06 risk of bias; 0.79 consistency; 0.21 precision; and 0.18 overall quality. Assessors agreed in 100 % of cases for directness. Precision created the most uncertainty due to difficulties in identifying ‘‘optimal’ ’ information size and ‘‘clinica

Interventions for behaviour change and self-management in stroke secondary prevention: protocol for an overview of reviews

Abstract Background Stroke secondary prevention guidelines recommend medication prescription and adherence, active education and behavioural counselling regarding lifestyle risk factors. To impact on recurrent vascular events, positive behaviour/s must be adopted and sustained as a lifestyle choice, requiring theoretically informed behaviour change and self-management interventions. A growing number of systematic reviews have addressed complex interventions in stroke secondary prevention. Differing terminology, inclusion criteria and overlap of studies between reviews makes the mechanism/s that affect positive change difficult to identify or replicate clinically. Adopting a two-phase approach, this overview will firstly comprehensively summarise systematic reviews in this area and secondly identify and synthesise primary studies in these reviews which provide person-centred, theoretically informed interventions for stroke secondary prevention. Methods An overview of reviews will be conducted using a systematic search strategy across the Cochrane Database of Systematic Reviews, PubMed and Epistomonikas. Inclusion criteria: systematic reviews where the population comprises individuals post-stroke or TIA and where data relating to person-centred risk reduction are synthesised for evidence of efficacy when compared to standard care or no intervention. Primary outcomes of interest include mortality, recurrent stroke and other cardiovascular events. In phase 1, two reviewers will independently (1) assess the eligibility of identified reviews for inclusion; (2) rate the quality of included reviews using the ROBIS tool; (3) identify unique primary studies and overlap between reviews; (4) summarise the published evidence supporting person-centred behavioural change and self-management interventions in stroke secondary prevention and (5) identify evidence gaps in this field. In phase 2, two independent reviewers will (1) examine person-centred, primary studies in each review using the Template for Intervention Description and Replication (TIDieR checklist), itemising, where present, theoretical frameworks underpinning interventions; (2) group studies employing theoretically informed interventions by the intervention delivered and by the outcomes reported (3) apply GRADE quality of evidence for each intervention by outcome/s identified from theoretically informed primary studies. Disagreement between reviewers at each process stage will be discussed and a third reviewer consulted. Discussion This overview will comprehensively bring together the best available evidence supporting person-centred, stroke secondary prevention strategies in an accessible format, identifying current knowledge gaps

Impact of quality of evidence on the strength of recommendations: an empirical study

<p>Abstract</p> <p>Background</p> <p>Evidence is necessary but not sufficient for decision-making, such as making recommendations by clinical practice guideline panels. However, the fundamental premise of evidence-based medicine (EBM) rests on the assumed link between the quality of evidence and "truth" and/or correctness in making guideline recommendations. If this assumption is accurate, then the quality of evidence ought to play a key role in making guideline recommendations. Surprisingly, and despite the widespread penetration of EBM in health care, there has been no empirical research to date investigating the impact of quality of evidence on the strength of recommendations made by guidelines panels.</p> <p>Methods</p> <p>The American Association of Blood Banking (AABB) has recently convened a 12 member panel to develop clinical practice guidelines (CPG) for the use of fresh-frozen plasma (FFP) for 6 different clinical indications. The panel was instructed that 4 factors should play a role in making recommendation: quality of evidence, uncertainty about the balance between desirable (benefits) and undesirable effects (harms), uncertainty or variability in values and preferences, and uncertainty about whether the intervention represents a wise use of resources (costs). Each member of the panel was asked to make his/her final judgments on the strength of recommendation and the overall quality of the body of evidence. "Voting" was anonymous and was based on the use of GRADE (Grading quality of evidence and strength of recommendations) system, which clearly distinguishes between quality of evidence and strength of recommendations.</p> <p>Results</p> <p>Despite the fact that many factors play role in formulating CPG recommendations, we show that when the quality of evidence is higher, the probability of making a strong recommendation for or against an intervention dramatically increases. Probability of making strong recommendation was 62% when evidence is "moderate", while it was only 23% and 13% when evidence was "low" or "very low", respectively.</p> <p>Conclusion</p> <p>We report the first empirical evaluation of the relationship between quality of evidence pertinent to a clinical question and strength of the corresponding guideline recommendations. Understanding the relationship between quality of evidence and probability of making (strong) recommendation has profound implications for the science of quality measurement in health care.</p

To what extent do site-based training, mentoring, and operational research improve district health system management and leadership in low- and middle-income countries: a systematic review protocol

District health managers play a key role in the effectiveness of decentralized health systems in low- and middle-income countries. Inadequate management and leadership skills often hamper their ability to improve quality of care and effectiveness of health service delivery. Nevertheless, significant investments have been made in capacity-building programmes based on site-based training, mentoring, and operational research. This systematic review aims to review the effectiveness of site-based training, mentoring, and operational research (or action research) on the improvement of district health system management and leadership. Our secondary objectives are to assess whether variations in composition or intensity of the intervention influence its effectiveness and to identify enabling and constraining contexts and underlying mechanisms